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ABSTRACT 

The  Humanitarian  Demining  Research  and  Development  Program  of  Night  Vision  and  Electronic  Sensors  Directorate 
(NVESD),  under  the  direction  of  the  Office  of  Assistant  Secretary  of  Defense  for  Special  Operations  and  Low-Intensity 
Conflict  (OASD/SOLIC)  and  with  participation  from  the  International  Test  and  Evaluation  Project  (ITEP)  for 
Humanitarian  Demining,  conducted  an  in-country  field  evaluation  of  HSTAMIDS  in  the  region  of  Humanitarian 
Demining  Unit  #1  (HMAU1)  in  Thailand.  Participants  included  the  US  Humanitarian  Demining  Team  of  NVESD, 
ITEP  personnel,  Thailand  Mine  Action  Center  (TMAC),  HALO  Trust  organization  from  Cambodia,  and  CyTerra 
Corporation.  The  primary  objectives  were  to  demonstrate  the  performance  of  the  U.S.  Army’s  latest  handheld 
multisensor  mine  detector,  the  AN/PSS-14,  in  a  demining  environment  in  comparison  to  the  performance  of  the  metal 
detector  being  used  by  the  local  deminers  and  also  to  assess  the  performance  of  the  trained  deminers  after  limited 
experience  and  training  with  the  HSTAMIDS. 
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1.  INTRODUCTION 

The  AN/PSS-14,  or  HSTAMIDS,  is  the  U.S.  Army’s  new  dual  sensor,  handheld  mine  detector  that  combines  an 
electromagnetic  induction  sensor,  ground  penetrating  radar  (GPR),  and  sophisticated  algorithms  to  detect  landmines 
while  rejecting  most  clutter.  The  HSTAMIDS  GPR  algorithms  give  it  the  discrimination  capabilities  that  metal 
detectors  alone  do  not  have.  The  Humanitarian  Demining  Research  and  Development  Program  of  Night  Vision  and 
Electronic  Sensors  Directorate  (NVESD),  under  the  direction  of  the  Office  of  Assistant  Secretary  of  Defense  for  Special 
Operations  and  Low-Intensity  Conflict  (OASD/SOLIC)  and  with  participation  from  the  International  Test  and 
Evaluation  Project  (ITEP)  for  Humanitarian  Demining,  conducted  an  in-country  field  evaluation  of  HSTAMIDS  in  the 
region  of  Humanitarian  Demining  Unit  #1  (HMAU1 )  in  Thailand. 

The  first  objective  of  this  field  evaluation  was  to  demonstrate  the  performance  of  the  HSTAMIDS  in  a  demining 
scenario.  This  was  accomplished  by  executing  a  comparative  evaluation  between  the  HSTAMIDS  and  a  metal  detector 
used  by  the  participating  demining  organization.  Both  systems  were  operated  by  experienced  operators.  The  second 
objective  was  to  train  the  demining  organization's  deminers  in  the  proper  use  and  operation  of  the  HSTAMIDS.  The 
third  objective  was  to  assess  the  performance  of  the  trained  deminers  after  limited  experience  and  training  with  the 
HSTAMIDS.  Participants  in  the  field  evaluation  included  the  US  Humanitarian  Demining  Team  of  NVESD,  ITEP 
personnel,  Thailand  Mine  Action  Center  (TMAC),  HALO  Trust  organization  from  Cambodia,  and  CyTerra 
Corporation. 

This  paper  describes  the  site  environment,  site  plan  and  setup,  training  and  test  procedures,  and  a  compilation  of  test 
results  along  with  brief  analysis  from  the  HSTAMIDS  field  evaluation  in  Thailand,  Sep-  Dec  2004. 
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2.  SITE  DESCRIPTION 


2.1  Location 

This  evaluation  was  conducted  near  the  minefields  at  the  Thailand  Mine  Action  Center's  (TMAC)  Humanitarian 
Demining  Action  Unit  (HMAU)  #1  in  Nongyakaeo,  Thailand.  Nongyakaeo  is  in  the  Sra  Kaeo  Province  of  Thailand 
located  approximately  260  km  east  of  Bangkok,  40  km  north  of  Aranyaprathet,  and  35  km  north  of  Paoy  Pet,  Cambodia. 
Active  demining  continues  in  the  surrounding  areas  of  HMAU  #1  and  along  the  Thai-Cambodian  border. 

2.2  Environment 

Thailand  has  a  very  humid,  tropical  climate  with  the  rainy  season  from  June  to  October  and  drier  weather  from 
November  to  April.  Heavy  rains  fell  on  the  evaluation  area  during  and  immediately  after  the  September/October  site 
setup.  This  helped  the  targets  to  weather-in,  but  also  may  have  moved  or  uncovered  more  metallic  clutter.  Only  one 
day  was  delayed  because  of  rain  during  the  November/December  evaluation. 

2.3  Soil  properties 

The  location  of  the  evaluation  site,  which  was  separated  into  a  test  and  training  area,  was  not  chosen  based  on  any 
particular  soil  type  or  characteristic,  but  rather  for  its  proximity  to  actual  minefields.  The  western  half  of  the  test  area 
appeared  to  be  clay  or  clay  loam1  with  a  short  layer  of  weed  like  vegetation.  The  eastern  half  had  a  high  concentration 
of  medium  pebbles1  that  made  the  soil  akin  to  concrete  and  difficult  to  dig.  The  training  area  was  separated  from  the 
test  area  by  about  50m.  Its  soil  was  similar,  but  appeared  to  have  a  higher  percentage  of  sand  and  silt  because  it  was  in 
the  drainage  area  of  the  field. 

Soil  electrical  conductivity  and  magnetic  susceptibility  were  measured  as  a  means  of  qualitative  comparison  between 
this  evaluation  and  other  test  events  even  though  sensor  performance  versus  environment  and  soil  properties  was  not 
one  of  the  test  metrics.  Measurements  were  taken  using  the  Geonics  EM38  soil  conductivity  meter  and  the  Bartington 
MS-2  magnetic  susceptibility  meter."  Conductivity  was  measured  with  the  EM38  in  both  vertical  and  horizontal 
orientations.  Table  1  contains  a  summary  of  the  soil  electromagnetic  properties. 


Table  1.  Summary  of  soil  electrical  conductivity  and  magnetic  susceptibility  measured  at  Thailand  test  site 


Range  of  values 

Average  over  site 

Conductivity: 

(Vertical) 

0  -  128mS/m 

65mS/m 

(Horizontal) 

7  -  129mS/m 

58mS/m 

Susceptibility 

31  -  196X  10 5  SI 

65  X  10 5  SI 

3.  RESOURCES 


3.1  Training  targets 

The  training  target  set  used  in  the  Thailand  evaluation  was  the  same  as  that  used  by  the  US  Army  for  all  AN/PSS-14 
training.  This  training  set  is  composed  almost  entirely  of  Simulant  Mines  (SIMs)  that  simulate  characteristics  found  in 
many  landmines,  but  that  do  not  represent  any  specific  mine.  SIMs  are  standard  test  targets  that  were  developed  by  the 
US  Army  Project  Manager  for  Mines,  Countermine  and  Demolitions  (now  known  as  PM  Close  Combat  Systems)  as 
part  of  a  four  nation  (FR,  GE,  UK,  US)  test  and  evaluation  working  group  establishing  International  Test  Operation 
Procedures  (ITOP)  for  Countermine  and  Humanitarian  Demining  equipment.3 


3.2  Test  targets 


The  test  target  set  was  composed  of  mines  that  are  found  in  the  area  of  HMAU  #1  and  mines  that  are  typically  used  for 
US  Army  testing.  Each  mine  was  characterized  in  one  of  the  following  four  categories  based  on  its  metal  content:  Anti- 
Tank,  Metallic  (AT-M);  Anti-Tank,  Low-Metal  (AT-LM);  Anti-Personnel,  Metallic  (AP-M);  and  Anti-Personnel,  Low- 
Metal  (AP-LM).  All  mines,  detonators,  and  fuses  were  free  from  explosive.  The  main  charges  were  replaced  with  RTV 
Silicone  Rubber  3110.  RTV  3110  closely  approximates  the  dielectric  constant  and  loss  tangent  of  explosives  and  is  the 
fill  used  in  SIMs.3  The  metal  components  and  characteristics  of  the  mines  remained  intact.  Linally,  each  mine  was 
inspected  for  accuracy  by  a  US  demolition  expert  and  a  well-known  mine  expert,  Colin  King,  from  the  UK. 

Within  the  four  categories  of  mines,  there  were  several  specific  types  of  mines.  To  get  statistically  significant  results, 
the  test  was  designed  so  that  most  mine  types  were  encountered  36  times.  Sometimes  this  was  not  possible  because  of  a 
limited  number  of  available  test  targets  in  the  resource  base.  In  the  case  of  mines  with  very  high  metal  content,  it  was 
assumed  that  all  detector  operators  would  find  these  mines  so  they  were  not  included  in  the  same  quantities  as  other 
mines.  Table  2  contains  the  list  of  test  targets  and  quantities  present  in  the  blind  test  lanes. 


Table  2.  Test  targets  and  quantities 


Mine  Category 

#  of  Mine  Types 

Quantity  of 
Mines 

Percentage 

AT-M 

2 

8 

7% 

AT-LM 

4 

34 

31% 

AP-M 

5 

38 

35% 

AP-LM 

4 

30 

27% 

(TOTAL) 

15 

110 

4.  SITE  PREPARATION  AND  SETUP 


4.1  General  site  preparations 

After  the  site  was  chosen,  extensive  preparations  were  completed  prior  to  site  setup  and  the  evaluation.  Brush  and 
vegetation  were  removed,  and  the  ground  was  leveled  to  facilitate  water  drainage.  A  vehicle-borne  magnet  was  used  to 
remove  significant  amounts  of  surface  metallic  clutter.  Linally,  a  three-rung,  wooden  fence  was  put  around  the  entire 
area  to  keep  cattle  out,  and  danger  signs  were  posted  to  discourage  people  from  entering  the  field. 

4.2  Training  area  setup 

The  training  lanes  were  setup  using  the  same  grid  design  that  is  used  in  US  Army  AN/PSS-14  training.  There  were 
eleven,  1.5m  by  15m  lanes  with  one  containing  a  subset  of  the  test  targets  (lane  K).  This  was  used  as  a  practice  lane  for 
the  trainees  to  traverse  prior  to  being  evaluated  in  the  test  area.  The  other  ten  lanes  contained  training  targets  (lanes  A  - 
J).  The  training  lane  layout  is  depicted  in  Figure  1. 
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I 
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Figure  1.  Training  lanes  A  -  J  and  practice  lane  K.  (Not  to  scale.) 

Each  lane  consisted  of  ten,  1.5m  by  1.5m  squares  marked  out  with  plastic  pegs.  A  1.5m  by  1.5m  grid  template  was  used 
to  locate  the  position  of  the  targets  within  each  square.  The  grid  template  was  made  of  PVC  pipe  with  string  equally 
dividing  it  into  a  10  by  10  matrix.  Its  horizontal  axis  was  denoted  with  letters  (A  to  J),  and  the  vertical  axis  was  labeled 
with  numbers  (1  to  10).  One  target  was  buried  per  square  in  the  lane.  The  grid  template  was  laid  over  the  plastic  pegs 
and  the  target  coordinates  were  noted. 

Prior  to  burial,  the  immediate  area  around  each  target  was  inspected  with  a  metal  detector  to  insure  that  it  was  free  from 
metallic  clutter.  All  targets  in  the  ten  training  lanes  were  buried  to  the  depths  specified  by  the  training  manual.  The  test 
targets  in  the  practice  lane  were  buried  at  the  same  depths  as  those  found  in  the  test  area. 

4.3  Test  site  setup 

The  test  area  consisted  of  ten,  lm  by  25m  blind  lanes  and  one,  lm  by  30m  calibration  lane.  The  test  field  layout  is 
shown  in  Figure  2.  Each  of  the  blind  lanes  was  5m  away  from  any  other  lane.  The  corners  of  each  were  marked  with 
non-metallic  pegs  and  had  string  running  between  them  along  the  edge  of  the  lane.  There  were  two  wooden  stakes,  one 
for  each  end  of  the  lane.  Each  stake  had  both  the  lane  number  and  directional  designation  for  the  end  of  the  lane  closest 
to  it  (Lane  1  East,  Lane  1  West,  etc). 


Calibration  Lane 


Lane  1 

Lane  2 

Lane  3 

Lane  4 
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Lane  10 

▲ 

North 


Figure  2.  Test  field  made  up  of  ten  blind  lanes  and  one  calibration  lane.  (Not  to  scale.) 

After  the  lane  corners  were  laid  out,  metal  detectors  were  used  to  locate  all  indigenous  metallic  clutter  in  the  lanes.  The 
test  targets  were  then  arranged  throughout  the  lane  so  that  they  had  sufficient  separation  between  them  and  the  clutter. 
This  was  done  to  insure  that  low  metal  mines  especially  were  not  placed  in  proximity  to  clutter  signals  that  would  give 


the  false  impression  of  detection.  No  indigenous  clutter  was  removed  from  the  lanes  after  being  located  by  the  metal 
detectors. 

Varying  the  burial  depth  of  the  mines  was  discussed  during  test  planning,  and  it  was  decided  with  the  ITEP  members 
that  depth  should  be  held  constant  to  limit  variables  and  increase  the  statistical  significance  of  the  test.  The  mine  depths 
were  agreed  upon  and  established.  All  antipersonnel  mines  were  buried  5cm  deep,  and  all  antitank  mines  were  buried 
10cm  deep  (measured  from  the  top  of  the  mine). 

In  addition  to  mines,  characterized  clutter  was  also  placed  in  the  lanes  to  better  understand  sensor  performance.  The 
clutter  consisted  of  mine  shrapnel,  fragments  of  miscellaneous  detonated  mines,  and  pieces  of  mine  safety  clips. 

After  the  indigenous  clutter  was  located  and  all  targets  were  emplaced,  the  entire  site  was  surveyed  with  a  laser 
surveying  system  to  establish  the  ground  truth  containing  lane  corner  points,  mines,  indigenous  clutter,  and 
characterized  clutter.  Fixed  concrete  reference  monuments  were  placed  around  the  test  field  for  calibration  and 
orientation  of  the  surveying  equipment. 

At  the  end  of  site  setup,  all  lanes  were  raked  so  that  visual  clues  were  removed.  The  site  then  remained  untouched  for 
five  weeks  prior  to  the  evaluation  to  allow  time  for  everything  to  weather-in. 


5.  METHODOLOGY 


5.1  Overview 

Testing  was  split  into  two  portions.  During  the  first  half  of  testing,  two  experienced  HSTAMIDS  operators  from  the  US 
and  two  experienced  metal  detector  operators  from  TMAC  tested  on  the  ten  blind  lanes.  This  took  place  concurrently 
with  the  HSTAMIDS  training  described  in  the  previous  section.  During  the  second  half  of  the  testing,  the  newly  trained 
HSTAMIDS  operators  from  TMAC  and  HALO  Trust  tested  on  the  same  ten  blind  lanes.  Thus,  there  were  three 
categories  of  operators:  Metal  Detector,  Experienced  HSTAMIDS,  and  HSTAMIDS  Trainees. 

The  objective  of  the  first  half  of  the  test  was  not  to  compare  the  HSTAMIDS  to  the  performance  of  all  metal  detectors. 
Rather,  to  establish  a  baseline  performance  with  the  local  demining  technology  and  a  baseline  performance  of  the 
HSTAMIDS  in  the  hands  of  experienced  operators,  both  as  a  means  of  comparison  with  the  performance  of  the  newly 
trained  HSTAMIDS  operators.  The  TMAC  deminers  tested  with  the  same  metal  detector,  the  Vallon  VMH-1,  which 
they  use  during  demining  operations  at  HMAU  #1. 


5.2  Training 

Five  deminers — two  from  TMAC  and  three  from  HALO  Trust,  Cambodia — were  trained  during  this  evaluation  by  the 
same  US  contractor  that  provides  AN/PSS-14  training  to  the  US  Army  and  Marine  Corps.  All  training  was  done 
through  interpreters.  This  added  an  additional  challenge  because  the  interpreters  had  no  experience  in  demining 
practices  or  technology.  Maintenance  and  troubleshooting  techniques  were  not  covered  during  this  training  interval  to 
allow  for  more  hands-on  training  time. 

The  primary  focus  of  the  deminer  training  was  HSTAMIDS  proficiency  and  target  detection.  As  in  all  HSTAMIDS 
training,  only  basic  clutter  rejection  was  taught.  This  is  a  skill  that  becomes  enhanced  with  continued  experience  using 
the  HSTAMIDS. 

The  trainers  had  the  HSTAMIDS  trainees  go  over  the  training  lanes  as  if  they  were  actual  minefields.  The  trainees  used 
non-metallic  markers  (poker  chips)  to  mark  all  detections.  The  trainers  then  used  the  grid  template  to  score  the  trainees 
by  comparing  the  coordinates  of  the  poker  chips  to  the  coordinates  of  the  ground  truth  and  noting  which  targets  were 
found  and  which  were  missed.  Proper  sweep  speed,  height,  and  coverage  were  taught  and  then  carefully  observed. 


To  prepare  the  trainees  for  the  blind  test  when  they  would  be  encountering  inert  mines  instead  of  SIMs,  the  trainers  had 
them  go  over  the  practice  lane  in  the  training  area  and  the  calibration  lane  in  the  test  area.  This  preparation  was  limited 
to  only  a  few  hours  at  the  end  of  regular  training. 

5.2  Test  procedures 

During  testing,  the  operators  went  down  each  lane  until  a  detection  was  encountered.  Non-metallic  poker  chips  were 
then  used  to  mark  detections.  Both  the  metal  detector  operators  and  HSTAMIDS  operators  marked  every  thing  they 
detected.  The  metal  detector  operators  marked  every  signal  they  encountered  as  a  suspected  mine  according  to 
worldwide  standard  operating  procedures.  The  HSTAMIDS  operators  made  a  determination  whether  the  target  was  a 
mine  or  clutter,  and  marked  each  detection  with  one  of  two  colors  of  chips.  Red  chips  for  mines  and  white  for  clutter. 

Data  collectors  assisted  the  operators  in  placing  a  chip  on  each  detection  and  kept  track  of  how  many  had  been  placed. 
Once  a  lane  was  completed,  a  laser  surveying  system  was  used  to  record  the  location  of  all  chips.  This  data  formed  an 
alarm  file  for  that  particular  lane  run.  Each  alarm  file  was  input  into  Countermine  Test  Management  Software  (CTMS) 
to  compare  operator  detections  versus  the  original  ground  truth  for  that  lane.  CTMS  was  used  to  provide  near  real-time 
test  statistics  for  each  operator  and  verify  surveying  results  before  any  chips  were  removed  from  the  test  lanes. 

To  acquire  a  statistically  significant  number  of  mine  encounters,  each  category  of  operator  had  to  cover  the  test  lanes  a 
total  of  four  times.  The  two  experienced  HSTAMIDS  operators  and  two  metal  detector  operators  traversed  each  lane 
twice,  once  in  each  direction.  No  operator  was  to  go  over  a  lane  more  than  once  in  the  same  day.  For  the  five 
HSTAMIDS  trainees  the  lanes  were  divided  up  so  that  they  covered  the  site  four  times  as  a  whole,  but  no  individual 
deminer  went  over  a  test  lane  more  than  once. 

5.3  Test  metrics 

The  HSTAMIDS  Thailand  test  results  were  scored  according  to  the  following  description  of  detections  and  false  alarms. 
Operators  were  credited  with  a  detection  if  their  chip  was  within  a  certain  distance,  /?hai0,  of  the  edge  of  a  target.  Any 
chip  outside  of  this  distance  was  considered  a  false  alarm.  If  more  than  one  chip  fell  within  the  detection  halo,  the 
operator  was  only  credited  with  a  single  detection.  The  value  of  Rh.dio  was  15cm  during  this  evaluation  as  in  all  US 
Army  testing  of  the  HSTAMIDS.  Figure  3  illustrates  the  possible  outcomes  for  detection  and  false  alarms. 


Figure  3.  Pictorial  definition  of  detections  and  false  alarms.4 

The  detection  probability  (Pd)  and  false  alarm  rate  (FAR)  are  the  two  primary  measures  derived  from  this  type  of  test. 
The  detection  probability  is  the  fraction  of  the  encountered  mines  that  are  detected: 

#  of  mines  detected 


#  of  mines  encountered 


(1) 


The  false-alarm  rate  has  been  commonly  defined  in  the  test  community  as  the  number  of  false  alarms  per  square  meter 
of  test  lane: 


_ .  „  #  of  false  alarms 

FAR  = - . 

total  lane  area 

The  probability  of  false  positive  (Pfp)  is  another  measure  that  can  be  obtained, 
positive  is  defined  as  the  fraction  of  characterized  clutter  that  is  detected: 


(2) 

For  this  test,  the  probability  of  false 


Pfr  = 


#  of  characterized  clutter  detected 
#  ofcharacterized  clutter  encountered 


(3) 


6.  TEST  RESULTS 


6.1  General  test  information 

These  are  the  test  results  for  the  three  categories  of  operators:  metal  detector,  experienced  HSTAMIDS,  and  TMAC 
and  HALO  Trust  HSTAMIDS  trainees.  Table  3  gives  some  general  information  for  each  operator  category.  It  contains 
the  number  of  lanes  that  each  operator  covered,  total  area  covered,  number  of  mines  encountered,  and  number  of 
emplaced,  characterized  clutter  encountered.  Table  4  contains  the  number  of  mine  encounters  per  mine  category. 
(*During  the  test,  one  metal  detector  operator  did  an  extra  lane.  This  is  why  the  numbers  are  not  identical  in  Table  3  or 
in  Table  4  for  all  three  operator  categories.) 

Table  3.  General  test  information  for  each  operator  category 


#  of 
Lanes 

Area 

Covered 

(m2) 

#  of  Mines 
Encountered 

#  of  Emplaced 
Clutter  Encountered 

Metal  Detector* 

41 

1023 

450 

488 

Experienced 

HSTAMIDS 

40 

998 

440 

472 

HSTAMIDS 

Trainees 

40 

998 

440 

472 

Table  4.  Number  of  mine  encounters  per  mine  category 


AT-M 

AT-LM 

AP-M 

AP-LM 

Encounters 

Encounters 

Encounters 

Encounters 

Metal  Detector* 

33 

141 

155 

121 

Experienced 

HSTAMIDS 

32 

136 

152 

120 

HSTAMIDS 

Trainees 

32 

136 

152 

120 

6.2  Results  for  metal  detector  operators 

Table  5  shows  the  false  alarm  rate,  probability  of  detection,  and  probability  of  false  positive  for  the  metal  detector 
operators.  The  total  false  alarm  rate  includes  indigenous  and  characterized  clutter. 


Table  5.  Test  results  for  the  Vallon  VMH-1  metal  detector  operators 


avg. 

FAR  (m'2) 

90%  CL  range 

Total 

1.023 

1.078  -0.968 

Indigenous 

0.598 

0.641  -  0.555 

PfP 

avg. 

90%  CL  range 

Characterized 

Clutter 

0.82 

0.85  -  0.79 

avg. 

P, 

90%  CL  range 

AT-M 

1.00 

1-0.91 

AP-M 

0.98 

0.99  -  0.95 

AT-LM 

0.29 

0.36  -  0.23 

AP-LM 

0.96 

0.98  -  0.92 

The  overall  detection  rate  for  the  metal  detector  operators  was  76%.  They  did  well  with  the  AT-M  and  both  categories 
of  AP  mine,  but  the  metal  detector  sensitivity  to  low  metallic  anti-tank  mines  was  poor.  The  probability  of  false 
positives  is  essentially  a  detection  probability  for  clutter.  The  results  show  that  only  82%  of  the  characterized  clutter 
that  was  added  to  the  lanes  was  detected  by  the  metal  detector  operators. 

6.3  Results  for  experienced  HSTAMIDS  operators 

The  following  tables  provide  the  results  for  the  experienced  HSTAMIDS  operators.  The  results  are  split  into  two 
categories:  Probability  of  Alert  and  Probability  of  Detection  with  Correct  Classification.  The  probability  of  alert,  shown 
in  Table  6- A,  is  based  on  the  combination  of  what  the  experienced  HSTAMIDS  operators  identified  as  mines  plus  what 
they  identified  as  clutter.  These  results  effectively  show  the  total  amount  of  what  was  detected  before  any 
discrimination.  The  functionality  of  the  probability  of  alert  for  the  HSTAMIDS  is  the  same  as  the  probability  of 
detection  for  the  metal  detector  since  the  metal  detector  cannot  do  any  discrimination. 

Table  6-A.  Probability  of  Alert  (mine  and  clutter  declarations)  for  the  experienced  HSTAMIDS  operators 


avg. 

FAR  (m'2) 

90%  CL  range 

Total 

0.884 

0.935  -0.832 

Indigenous 

0.417 

0.453  -0.381 

PfP 

avg. 

90%  CL  range 

Characterized 

Clutter 

0.99 

1  -  0.98 

avg. 

P, 

90%  CL  range 

AT-M 

1.00 

1-0.91 

AP-M 

1.00 

1  -  0.98 

AT-LM 

1.00 

1  -  0.98 

AP-LM 

0.99 

1  -  0.96 

The  overall  probability  of  alert  was  99.7%  for  the  experienced  HSTAMIDS  operators.  One  mine  out  of  440  encounters 
was  missed.  (There  was  a  mine  declaration  right  beside  this  target,  but  it  was  2cm  outside  the  detection  halo  and  thus 
counted  as  a  false  alarm.)  These  results  are  significantly  better  than  the  metal  detector  operators'  detection  rate  of  76%. 
Nearly  all  of  the  characterized  clutter  was  also  detected  as  shown  by  the  Pfp. 


The  probability  of  detection  with  correct  classification,  shown  in  Table  6-B,  is  based  on  what  the  experienced 
HSTAMIDS  operators  identified  as  mines.  These  results  show  the  discrimination  that  can  be  achieved  with  the 
HSTAMIDS  especially  when  comparing  the  false  alarm  rate  and  probability  of  false  positive  to  the  results  in  Table  6-A. 

Table  6-B.  Probability  of  Detection  with  Correct  Classification  (mine  declarations  only)  for  the  experienced 
HSTAMIDS  operators 


avg. 

FAR  (m'2) 

90%  CL  range 

Total 

0.201 

0.225  -0.176 

Indigenous 

0.124 

0.144-0.104 

Pfr 

avg. 

90%  CL  range 

Characterized 

Clutter 

0.19 

0.27-0.12 

avg. 

P, 

90%  CL  range 

AT-M 

1.00 

1-0.91 

AP-M 

0.95 

0.98  -  0.92 

AT-LM 

0.93 

0.97  -  0.89 

AP-LM 

0.93 

0.96  -  0.87 

The  experienced  HSTAMIDS  operators  had  a  high  probability  of  detection  with  correct  classification  (94%  overall) 
while  significantly  reducing  the  false  alarm  rate.  They  were  able  to  correctly  classify  81%  of  the  emplaced  clutter  as 
clutter.  Looking  at  the  total  false  alarm  rate,  which  includes  all  of  the  indigenous  clutter  that  remained  in  the  lanes,  the 
experienced  HSTAMIDS  operators  achieved  77%  overall  clutter  rejection. 

6.4  Results  for  HSTAMIDS  trainees 

The  following  tables  provide  the  results  for  the  TMAC  and  HALO  Trust  HSTAMIDS  trainees.  Table  7- A  shows  the 
probability  of  alert  results  before  discrimination. 

Table  7-A.  Probability  of  Alert  (mine  and  clutter  declarations)  for  the  HSTAMIDS  trainees 


avg. 

FAR  (m'2) 

90%  CL  range 

Total 

0.782 

0.830  -0.733 

Indigenous 

0.354 

0.387  -0.320 

Pfr 

avg. 

90%  CL  range 

Characterized 

Clutter 

0.91 

0.93  -  0.88 

avg. 

P i 

90%  CL  range 

AT-M 

1.00 

1-0.91 

AP-M 

1.00 

1  -  0.98 

AT-LM 

0.97 

0.99  -  0.93 

AP-LM 

0.97 

0.99  -  0.93 

Just  as  the  experienced  operators,  the  trainees  outperformed  the  metal  detector  operators  at  finding  mines.  The  overall 
probability  of  alert  was  98%  for  the  HSTAMIDS  trainees.  They  also  detected  most  of  the  characterized  clutter. 


Table  7-B  contains  the  probability  of  detection  with  correct  classification  after  the  determination  was  made  whether  the 
target  was  mine  or  clutter.  This  was  a  unique  experience  for  the  trainees  since  deminers  in  the  field  cannot  make  this 
decision  with  the  equipment  currently  available  to  them. 

Table  7-B.  Probability  of  Detection  with  Correct  Classification  (mine  declarations  only)  for  the  HSTAMIDS  trainees 


avg. 

FAR  (in2) 

90%  CL  range 

Total 

0.254 

0.282  -  0.227 

Indigenous 

0.128 

0.149-0.108 

Pfr 

avg. 

90%  CL  range 

Characterized 

Clutter 

0.30 

0.34  -  0.26 

avg. 

Pc 

90%  CL  range 

AT-M 

1.00 

1-0.91 

AP-M 

0.95 

0.98  -  0.91 

AT-LM 

0.78 

0.84-0.71 

AP-LM 

0.80 

0.86  -  0.73 

The  trainees  did  not  do  quite  as  well  at  correctly  identifying  the  mines  as  the  experienced  operators  did,  but  their  clutter 
rejection  performance  was  very  good  considering  their  lack  of  experience.  The  overall  probability  of  detection  with 
correct  classification  was  86%  for  the  trainees,  and  they  reduced  the  total  FAR  by  about  68%.  Their  clutter  rejection 
results  were  nearly  as  good  as  the  experienced  operators  clutter  rejection  rate  of  77/%. 

Language  barriers,  lack  of  experience  in  test  scenarios,  and  limited  HSTAMIDS  experience  were  all  factors  that 
contributed  to  the  trainees’  lower  performance  when  compared  to  the  experienced  operators'  results.  However,  they 
still  showed  an  improvement  over  the  performance  of  the  metal  detector  operators  in  overall  Pd  and  FAR,  even  in  the 
discrimination  mode. 

6.5  Time  comparison 

During  a  one-month  period  in  2003  at  FIMAU  #1,  7  AP  mines  and  approximately  4,500  metal  fragments  were  removed 
from  the  ground  in  a  11,872  square  meter  area.  TMAC  data  shows  that  the  average  time  to  determine  whether  a 
detected  signal  is  a  mine  or  a  metal  fragment  is  20  to  30  minutes.  In  Table  8,  the  lower  estimate  of  20  minutes  is  used 
to  show  the  potential  time  savings  that  the  HSTAMIDS  would  bring  to  TMAC’s  demining  operation. 


The  time  savings  was  calculated  using  the  difference  between  metal  detector  and  HSTAMIDS  total  time.  Total  time 
equaled  time  scanning  plus  time  digging.  The  scan  rate  was  calculated  for  each  operator  category  by  keeping  track  of 
how  long  it  took  the  operators  to  finish  each  lane. 


Table  8.  Clutter  discrimination  and  potential  time  savings  (Calculated  with  TMAC  records  showing  20  minutes  to 
determine  mine  or  clutter) 


#  Clutter 
Detected 

#  Clutter 
Called 
Mine 

Time 

Scanning 

Time  Digging 
Suspected  Mines 
(hours) 

Time  Savings 
(hours) 

Metal 

Detector 

956 

956 

17  hours  @ 
1.01  min/m2 

319 

0 

Experienced 

HSTAMIDS 

806 

183 

21  hours  @ 
1.25  min/m2 

61 

254 

HSTAMIDS 

Trainees 

713 

232 

38  hours  @ 
2.29  min/m2 

77 

221 

In  the  hands  of  experienced  operators,  the  HSTAMIDS  showed  a  time  savings  of  over  250  man-hours  while 
maintaining  an  overall  probability  of  detection  of  94%. 


7.  CONCLUSIONS 

The  objectives  of  the  HSTAMIDS  field  evaluation  in  Thailand  were  successfully  accomplished.  The  performance  of 
the  HSTAMIDS  was  demonstrated,  local  deminers  were  trained  in  its  use,  and  the  deminers’  initial  capabilities  with  the 
system  were  evaluated. 

This  test  was  challenging  in  several  respects.  The  mines  were  buried  at  deeper  depths  than  what  is  typically  found  in 
real-world  conditions.  Clutter  density  was  much  greater  than  most  metal  detector  tests  that  try  to  minimize  metallic 
clutter  in  the  lanes.  Finally,  a  test  with  15  different  mines,  many  of  which  are  the  most  challenging  in  the  world  to 
detect,  makes  it  very  difficult  for  operators  to  determine  mine  versus  clutter  with  such  a  wide  variety  of  potential 
targets. 

Despite  the  challenges,  the  HSTAMIDS  was  clearly  better  than  the  metal  detector  in  finding  the  mines.  The 
experienced  HSTAMIDS  operators  found  all  of  the  mines  but  one  while  the  metal  detector  operators  missed  over  100 
out  of  440  mines.  When  scoring  what  the  experienced  HSTAMIDS  operators  identified  as  mines,  they  rejected  77%  of 
the  false  alarms  with  an  overall  detection  rate  of  94%.  Even  with  limited  experience,  the  trainees  did  almost  as  well  as 
the  experienced  HSTAMIDS  operators,  rejecting  the  majority  of  the  clutter  as  they  correctly  classified  86%  of  the 
mines.  When  considering  only  the  types  of  mines  that  are  found  at  HMAU#1,  the  HSTAMIDS  shows  the  ability  to 
detect  100%  while  providing  significant  clutter  rejection  and  time  savings.  Thus  the  HSTAMIDS  could  provide  real 
benefit  to  the  demining  operations  in  this  area  of  the  world. 

The  Humanitarian  Demining  R&D  team  continues  to  focus  on  projects  to  improve  the  discrimination  performance  of 
the  HSTAMIDS — projects  to  help  make  the  operator’s  decision  easier  so  that  he  can  maintain  detections  while  reducing 
false  alarms.  Continuing  work  on  algorithm  development  and  hardware  modifications  is  showing  good  results  and 
promise  of  further  improvements  for  the  HSTAMIDS. 


ACKNOWLEDGEMENTS 

Grateful  acknowledgements  to  the  ITEP  organizations  and  representatives  from  Canada,  the  Netherlands,  Sweden,  and 
the  United  Kingdom  for  their  participation  and  assistance  in  every  aspect  of  this  evaluation.  Thanks  to  TMAC  for 
providing  the  site  and  to  both  TMAC  and  HALO  Trust  for  allowing  some  of  their  deminers  to  participate.  Thanks  to 
JUSMAG  and  the  US  Embassy  in  Thailand  for  their  assistance,  interest,  and  support.  Thanks  to  S-3  Services  logistics 
and  interpreter  support  in-country.  Thanks  to  the  Institute  for  Defense  Analyses.  And  finally,  thanks  to  all  the 
individuals  at  Fort  AP  Hill,  NVESD  Countermine,  and  Humanitarian  Demining  who  worked  hard  to  make  this 
evaluation  a  success. 


REFERENCES 


1.  Online  Glossary  of  Soil  Science  Terms.  Soil  Science  Society  of  America  Web  site.  Available  at: 
http://www.soils.org/sssagloss/index.html.  Accessed  8  February  2005. 

2.  Reidy,  D.M.,  R.  Walls,  C.  Lee,  “Effects  of  soil  magnetic  susceptibility  and  electrical  conductivity  on 
electromagnetic  detection  of  landmines,”  Detection  and  Remediation  Technologies  for  Mines  and  Minelike 
Targets  IX,  edited  by  R.S.  Harmon,  J.T.  Broach,  J.H.  Holloway,  Vol  5415,  pp.  658  -664,  SPIE,  Bellingham,  WA, 
2004. 

3.  Scientific  and  technical  report  page  for  simulant  mines.  Department  of  Defense's  Unexploded  Ordnance  Center  of 
Excellence  (UXOCOE)  Web  site.  Available  at:  http://www.uxocoe.brtrc.com/techlibrary/TechRpts/miscl.asp. 
Accessed  8  February  2005. 

4.  Rosen.  E.,  E.  Ayers,  F.  Rotondo,  J.  Teichman,  “Results  of  the  Humanitarian  Demining  Sensors  Field  Test  at 
Aberdeen  Proving  Ground,  November  2002,”  Institute  for  Defense  Analyses  Document  D-2827,  pp.  16,  17,  April 
2003. 


